Sending GET and POST Requests Using the Requests Library in Python

Course: Python Training

Topic: Sending GET and POST Requests Using the Requests Library

Course Link: Python Training Course

Demo Link: Register for a Course Demo

Website: JustAcademy Official Website

1. Introduction to the Requests Library

The Requests library is a popular Python library used to send HTTP requests to web servers and APIs. It allows Python applications to communicate with websites, retrieve information, submit data, integrate third-party services, and interact with REST APIs.

When a Python application needs to retrieve weather information, submit registration details, fetch product information, or communicate with an external service, it can use Requests to send HTTP requests and process the responses.

Two of the most commonly used HTTP methods are GET and POST. GET is primarily used to retrieve data, while POST is commonly used to submit data to a server for processing or resource creation.

Learning Objectives

• Understand HTTP communication and the Requests library.
• Install Requests and configure a Python environment.
• Send GET requests to web servers and REST APIs.
• Pass query parameters to GET requests.
• Send POST requests using form data and JSON payloads.
• Read HTTP status codes, headers, and response bodies.
• Parse JSON responses into Python objects.
• Handle HTTP errors, connection errors, and timeouts.
• Use headers, authentication, and sessions.
• Test and troubleshoot API requests using Python.

2. What Is the Requests Library?

Requests is an HTTP client library that simplifies sending HTTP requests from Python programs. It handles many lower-level details, such as query-string encoding, connection management, response decoding, and common authentication mechanisms.

Without Requests, a developer would need to work with lower-level networking tools or other HTTP client libraries. Requests provides a straightforward interface for common tasks.

Key Features

• Supports GET, POST, PUT, PATCH, DELETE, and other HTTP methods.
• Supports query parameters and request headers.
• Can send JSON payloads and form-encoded data.
• Provides convenient access to response status codes and headers.
• Can parse JSON response bodies.
• Supports authentication, cookies, redirects, and sessions.
• Supports connection and read timeouts.
• Provides exceptions for handling network and HTTP errors.

3. Installing Requests

Before sending HTTP requests, install the Requests library in your Python environment.

Installation Command

python -m pip install requests

On some systems, you may need to use python3 instead of python.

Verify the Installation

import requests

print(requests.__version__)

If the installation is successful, the program displays the installed Requests version.

Import the Library

import requests

The requests module provides functions such as requests.get() and requests.post() for making HTTP requests.

4. Understanding HTTP Requests and Responses

HTTP stands for Hypertext Transfer Protocol. It defines how clients and servers exchange requests and responses over a network.

When Python sends a request to a server, the server processes it and returns a response. Requests represents that response through a Response object.

Request Components

Response Components

5. Understanding GET Requests

A GET request is used primarily to retrieve a resource or information from a server. It is one of the most common HTTP methods used when working with REST APIs.

For example, a Python program might send a GET request to retrieve a list of products, read student information, or obtain public information from an API.

Basic Syntax

requests.get(url, params=None, **kwargs)

Parameters:

• url: The endpoint to which the request is sent.
• params: Optional query parameters added to the URL.
• headers: Optional request headers.
• timeout: The time allowed for connection and response activity before a timeout exception occurs.
• auth: Optional authentication configuration.

Example: Sending a Basic GET Request

import requests

url = "https://jsonplaceholder.typicode.com/posts/1"

response = requests.get(url, timeout=10)

print("Status Code:", response.status_code)
print("Response Body:")
print(response.text)

This example requests a sample post from a public demonstration API. The server typically returns a JSON representation of the post.

Example Response

{
    "userId": 1,
    "id": 1,
    "title": "Example post title",
    "body": "Example post content"
}

The exact response content depends on the server. The example above illustrates the expected structure rather than guaranteeing the returned text.

6. Understanding the GET Response Object

The requests.get() function returns a Response object. This object provides access to the HTTP status code, response body, headers, and other information.

Important Response Attributes and Methods

Example: Inspecting a Response

import requests

url = "https://jsonplaceholder.typicode.com/posts/1"

response = requests.get(url, timeout=10)

print("Status:", response.status_code)
print("URL:", response.url)
print("Content Type:", response.headers.get("Content-Type"))
print("Response Text:", response.text)

This program displays important response details that can help with debugging and API integration.

7. Reading JSON Data from GET Requests

Many APIs return JSON data. Requests provides the response.json() method to parse a JSON response into a Python dictionary, list, or another compatible Python structure.

Example

import requests

url = "https://jsonplaceholder.typicode.com/posts/1"

response = requests.get(url, timeout=10)
response.raise_for_status()

post = response.json()

print("Post ID:", post["id"])
print("Title:", post["title"])
print("Body:", post["body"])

The response.json() method handles JSON parsing, but it does not itself confirm that the HTTP request succeeded. Checking the status or calling raise_for_status() is important before processing data as a successful result.

Common Uses of JSON Responses

• Displaying product information in a web application.
• Retrieving student or employee records.
• Fetching weather or other public API data.
• Reading account or order information from a backend service.
• Integrating Python scripts with third-party applications.

8. Sending Query Parameters with GET Requests

Query parameters allow a client to send additional values through a URL. They are commonly used for filtering, searching, sorting, and pagination.

Example Without the params Argument

import requests

url = "https://jsonplaceholder.typicode.com/posts?userId=1"

response = requests.get(url, timeout=10)
response.raise_for_status()

print(response.json())

Recommended Approach Using params

import requests

url = "https://jsonplaceholder.typicode.com/posts"

params = {
    "userId": 1
}

response = requests.get(
    url,
    params=params,
    timeout=10
)

response.raise_for_status()

print("Requested URL:", response.url)
print(response.json())

Using the params argument lets Requests construct the query string and encode parameter values appropriately.

Multiple Query Parameters

import requests

url = "https://example.com/api/products"

params = {
    "category": "electronics",
    "page": 2,
    "limit": 10,
    "sort": "price"
}

response = requests.get(
    url,
    params=params,
    timeout=10
)

print(response.url)

This is an illustrative example. Replace the endpoint with an actual API that supports these parameters before executing it.

9. Understanding POST Requests

A POST request submits data to a server for processing. It is commonly used to create a resource, register a user, submit a form, or initiate an operation.

Unlike a typical GET request, a POST request often contains a request body. The body may contain form-encoded data, JSON, or another format accepted by the endpoint.

Basic Syntax

requests.post(url, data=None, json=None, **kwargs)

Important arguments:

• url: The destination endpoint.
• data: Data to send in the request body, commonly form-encoded when a dictionary is provided.
• json: A Python object that Requests serializes to JSON.
• headers: Optional request headers.
• timeout: Timeout configuration.

Example: Sending a Basic POST Request

import requests

url = "https://jsonplaceholder.typicode.com/posts"

payload = {
    "title": "Learning Python",
    "body": "Practicing HTTP POST requests",
    "userId": 1
}

response = requests.post(
    url,
    json=payload,
    timeout=10
)

print("Status Code:", response.status_code)
print("Response:", response.text)

This example submits JSON data to a public demonstration API. The service commonly returns a representation of the submitted post with an assigned identifier. Demonstration APIs may simulate creation without permanently saving the record.

10. Sending JSON Data Using POST

The json argument is a convenient way to send JSON payloads. Requests serializes the supplied Python dictionary or other JSON-compatible object and sets the content type to application/json.

Example

import requests

url = "https://jsonplaceholder.typicode.com/posts"

student = {
    "name": "Priya Sharma",
    "email": "[email protected]",
    "course": "Python"
}

response = requests.post(
    url,
    json=student,
    timeout=10
)

print("Status Code:", response.status_code)
print("Response Headers:", response.headers)
print("Response JSON:", response.json())

Example JSON Payload

{
    "name": "Priya Sharma",
    "email": "[email protected]",
    "course": "Python"
}

When integrating with a real API, make sure the field names and data types match that API's documented schema.

11. Sending Form Data Using POST

Not every endpoint expects JSON. Traditional HTML forms and some APIs accept form-encoded data. Requests supports this format through the data argument.

Example

import requests

url = "https://httpbin.org/post"

form_data = {
    "username": "student01",
    "course": "Python",
    "level": "beginner"
}

response = requests.post(
    url,
    data=form_data,
    timeout=10
)

response.raise_for_status()

print(response.json())

When a dictionary is supplied through data, Requests typically encodes it as application/x-www-form-urlencoded. The example endpoint echoes information about the request for testing purposes.

JSON Versus Form Data

Choose the format required by the receiving endpoint. Do not assume that every POST endpoint accepts JSON.

12. Using Custom Headers in GET and POST Requests

HTTP headers carry information about a request. APIs often use headers to identify the content format, specify acceptable response formats, or provide authentication credentials.

Example: GET Request with Headers

import requests

url = "https://httpbin.org/headers"

headers = {
    "Accept": "application/json",
    "User-Agent": "PythonLearningClient/1.0"
}

response = requests.get(
    url,
    headers=headers,
    timeout=10
)

response.raise_for_status()

print(response.json())

Example: POST Request with Headers

import requests

url = "https://httpbin.org/post"

headers = {
    "Accept": "application/json",
    "Content-Type": "application/json"
}

payload = {
    "message": "Hello from Python"
}

response = requests.post(
    url,
    headers=headers,
    json=payload,
    timeout=10
)

response.raise_for_status()

print(response.json())

When using the json argument, Requests automatically sets the JSON content type. Setting the header explicitly is optional unless the API requires a specific value.

13. Understanding HTTP Status Codes

Status codes indicate the result of an HTTP request. Checking them helps distinguish successful responses from client errors, server errors, and other outcomes.

Example: Checking a Status Code

import requests

url = "https://jsonplaceholder.typicode.com/posts/1"

response = requests.get(url, timeout=10)

if response.status_code == 200:
    print("Request successful")
    print(response.json())
else:
    print("Request failed")
    print("Status:", response.status_code)

14. Handling HTTP Errors with raise_for_status()

Requests provides raise_for_status() to raise an HTTPError when a response indicates an unsuccessful HTTP status code, such as 404 or 500.

Example

import requests

url = "https://jsonplaceholder.typicode.com/posts/1"

try:
    response = requests.get(url, timeout=10)
    response.raise_for_status()

    data = response.json()
    print(data)

except requests.exceptions.HTTPError as error:
    print("HTTP error:", error)

except requests.exceptions.RequestException as error:
    print("Request failed:", error)

This approach separates HTTP errors from other request-related failures. The server may return a valid JSON error body even when the HTTP status indicates failure, so status checking remains important.

15. Handling Connection Errors and Timeouts

Network requests can fail for several reasons, including unavailable servers, DNS problems, network interruptions, and delayed responses. Requests provides exceptions to help handle these situations.

Example: Robust GET Request

import requests

url = "https://jsonplaceholder.typicode.com/posts/1"

try:
    response = requests.get(url, timeout=(3, 10))
    response.raise_for_status()

    data = response.json()
    print(data)

except requests.exceptions.Timeout:
    print("The request timed out.")

except requests.exceptions.ConnectionError:
    print("Could not connect to the server.")

except requests.exceptions.HTTPError as error:
    print("HTTP error:", error)

except requests.exceptions.RequestException as error:
    print("Unexpected request error:", error)

except requests.exceptions.JSONDecodeError:
    print("The response body is not valid JSON.")

The timeout tuple specifies a connection timeout and a read timeout, in seconds. A timeout is not necessarily a total deadline for the entire download; it limits waiting for connection or response activity.

Best practice: Set sensible timeouts on network requests. For retryable operations, use a carefully configured retry strategy. Do not automatically retry every POST request because the server might already have processed it.

16. Using Authentication with Requests

Some APIs require authentication before allowing access to protected resources. Requests supports several authentication mechanisms, depending on the API.

Basic Authentication Example

import requests

url = "https://httpbin.org/basic-auth/demo-user/demo-password"

response = requests.get(
    url,
    auth=("demo-user", "demo-password"),
    timeout=10
)

print("Status:", response.status_code)
print(response.text)

This is an illustrative example using a public testing endpoint. Do not use real credentials in examples, source code, or publicly shared files.

Bearer Token Authentication Example

import os
import requests

token = os.environ.get("API_TOKEN")

if not token:
    raise RuntimeError("API_TOKEN environment variable is missing")

url = "https://example.com/api/profile"

headers = {
    "Authorization": f"Bearer {token}",
    "Accept": "application/json"
}

response = requests.get(
    url,
    headers=headers,
    timeout=10
)

response.raise_for_status()
print(response.json())

Replace the example URL with the documented endpoint of the service you are using. Store credentials securely, use HTTPS, and never disable TLS certificate verification as a routine workaround.

17. Sending GET and POST Requests Using a Session

A Requests Session allows multiple requests to share configuration and cookies. It can also reuse underlying connections, which may improve efficiency when making repeated requests to the same service.

Example

import requests

with requests.Session() as session:
    session.headers.update({
        "Accept": "application/json",
        "User-Agent": "PythonCourseClient/1.0"
    })

    get_response = session.get(
        "https://jsonplaceholder.typicode.com/posts/1",
        timeout=10
    )
    get_response.raise_for_status()
    print("GET:", get_response.json())

    payload = {
        "title": "Session Example",
        "body": "Sending data using Requests",
        "userId": 1
    }

    post_response = session.post(
        "https://jsonplaceholder.typicode.com/posts",
        json=payload,
        timeout=10
    )
    post_response.raise_for_status()
    print("POST:", post_response.json())

The session shares configured headers across requests and is closed automatically when the with block ends.

18. Difference Between GET and POST Requests

GET requests can be cached when the request and caching rules permit it. POST requests are not generally cached by default. Sensitive data should not be placed in a URL merely because POST requests often use a body; HTTPS and suitable security controls are required for both methods.

19. Mini Project: Fetch and Submit Student Information

Build a small Python script that retrieves sample student information and submits a new sample record to a demonstration API.

Project Objectives

• Send a GET request to retrieve information.
• Parse a JSON response into a Python object.
• Send a POST request with a JSON payload.
• Check HTTP status codes.
• Handle request errors and timeouts.

Complete Example

import requests

BASE_URL = "https://jsonplaceholder.typicode.com"


def get_post(post_id):
    url = f"{BASE_URL}/posts/{post_id}"

    try:
        response = requests.get(url, timeout=10)
        response.raise_for_status()

        post = response.json()

        print("Retrieved Post")
        print("ID:", post["id"])
        print("Title:", post["title"])
        print("Body:", post["body"])

    except requests.exceptions.RequestException as error:
        print("GET request failed:", error)

    except requests.exceptions.JSONDecodeError:
        print("The server returned invalid JSON.")

    except KeyError as error:
        print("Expected field is missing:", error)


def create_post(title, body, user_id):
    url = f"{BASE_URL}/posts"

    payload = {
        "title": title,
        "body": body,
        "userId": user_id
    }

    try:
        response = requests.post(
            url,
            json=payload,
            timeout=10
        )
        response.raise_for_status()

        print("POST Status:", response.status_code)
        print("Created Post:", response.json())

    except requests.exceptions.RequestException as error:
        print("POST request failed:", error)

    except requests.exceptions.JSONDecodeError:
        print("The server returned invalid JSON.")


if __name__ == "__main__":
    get_post(1)

    create_post(
        "Learning Python Requests",
        "Practicing GET and POST requests",
        1
    )

Expected behavior: The first function retrieves a sample post. The second submits a new sample post and displays the returned response. The demonstration service may simulate creation without persistent storage.

20. Best Practices When Using Requests

• Set timeouts: Prevent a program from waiting indefinitely for network activity.
• Check HTTP status: Use raise_for_status() or explicitly check expected status codes.
• Handle exceptions: Account for timeouts, connection failures, HTTP errors, and invalid JSON.
• Use the correct payload format: Choose json= or data= according to the API contract.
• Keep credentials secure: Use environment variables or an appropriate secret manager.
• Use HTTPS: Protect data in transit and retain certificate verification.
• Respect rate limits: Follow service usage policies and honor retry guidance such as the Retry-After header when applicable.
• Retry carefully: Use backoff for suitable transient failures and avoid unsafe automatic retries of non-idempotent operations.
• Validate response data: Do not assume every response contains the fields your program expects.
• Use sessions for repeated calls: Reuse configuration and connections when appropriate.
• Avoid excessive logging: Do not print passwords, access tokens, or sensitive personal data.
• Read API documentation: Verify endpoints, required headers, payload schemas, authentication, and response codes.

21. Common Errors and Troubleshooting

22. Practical Exercises

1. Basic GET Request: Retrieve a sample post and print its ID, title, and body.
2. GET with Parameters: Fetch posts belonging to a specified user using the params argument.
3. JSON Parsing: Convert a JSON response into a Python dictionary and access selected values.
4. Basic POST Request: Submit a title and body to a public testing API using json=.
5. Form Submission: Send key-value form data using the data argument.
6. Custom Headers: Add an Accept header and inspect the response headers.
7. Error Handling: Request a nonexistent resource and handle the resulting HTTP error.
8. Timeout Handling: Configure a timeout and handle a timeout exception.
9. Session Practice: Send multiple requests through a shared Requests Session.
10. Mini Project: Build a command-line program that retrieves and submits sample product information.

23. Interview Questions and Answers

Q1. What is the Requests library in Python?

Answer: Requests is a Python HTTP client library that makes it easy to send HTTP requests, interact with APIs, and process server responses.

Q2. How do you send a GET request?

Answer: Import Requests and call requests.get(url). In production code, use a suitable timeout and check the HTTP response status.

Q3. How do you send a POST request?

Answer: Use requests.post(url, json=payload) to send a JSON payload or requests.post(url, data=form_data) for form-encoded data.

Q4. What is the difference between data= and json=?

Answer: The data= argument sends supplied data, commonly as form-encoded fields when given a dictionary. The json= argument serializes a JSON-compatible Python object and sets the JSON content type.

Q5. What does response.json() do?

Answer: It parses a JSON response body into a Python object, such as a dictionary or list. It raises a JSON decoding exception if the response body is not valid JSON.

Q6. What is response.status_code?

Answer: It returns the HTTP status code provided by the server, such as 200, 201, 404, or 500.

Q7. What does raise_for_status() do?

Answer: It raises an HTTPError when the response status indicates an unsuccessful HTTP request, generally for 4xx and 5xx status codes.

Q8. Why should a timeout be specified?

Answer: A timeout limits how long Requests waits for connection or response activity, preventing indefinite waits caused by an unresponsive server or network.

Q9. What is the purpose of query parameters?

Answer: Query parameters provide additional values for filtering, searching, sorting, or paginating results. Requests can encode them using the params argument.

Q10. What is a Requests Session?

Answer: A Session stores configuration and cookies across requests and can reuse connections, making repeated requests to a service more efficient.

Q11. What is the difference between response.text and response.content?

Answer: response.text provides the response body as decoded text, while response.content provides the response body as bytes.

Q12. How do you handle a network failure?

Answer: Use exception handling for Requests exceptions, including connection failures and timeouts, and log or report a meaningful error without exposing sensitive information.

Q13. Can a POST request return a 200 status code?

Answer: Yes. The response status depends on the server's behavior. A successful POST may return 200 OK, 201 Created, or another appropriate status code.

Q14. Does successfully parsing JSON mean that a request succeeded?

Answer: No. An error response can contain valid JSON. Check the HTTP status separately before treating the response as successful.

Q15. How can you protect API credentials in Python?

Answer: Store credentials in environment variables or a secret manager, use HTTPS, restrict access, and avoid hardcoding or logging secrets.

24. Quick Revision Checklist

• Install Requests using python -m pip install requests.
• Import the library using import requests.
• Use requests.get() to retrieve resources.
• Use params= to provide query parameters.
• Use requests.post() to submit data.
• Use json= for JSON request bodies.
• Use data= for form-encoded data when appropriate.
• Inspect response.status_code, response.headers, and response.text.
• Use response.json() to parse valid JSON responses.
• Call response.raise_for_status() to detect HTTP errors.
• Configure timeouts for network requests.
• Handle connection failures and invalid response data.
• Use sessions for repeated requests when appropriate.
• Protect credentials and follow the API's documented requirements.

25. Conclusion

The Requests library makes HTTP communication straightforward in Python. GET requests allow applications to retrieve information, while POST requests enable them to submit data for processing or resource creation.

By learning how to pass query parameters, send JSON payloads, inspect response objects, handle HTTP status codes, configure timeouts, and manage exceptions, developers can build reliable integrations with REST APIs and external services.

Continue learning through the Python Training Course at JustAcademy. Visit the JustAcademy Official Website or register for a course demo to explore further learning opportunities.